Scale-up of a novel vital signs alert device to improve maternity care in Sierra Leone: a mixed methods evaluation of adoption

Background The CRADLE (Community blood pressure monitoring in Rural Africa: Detection of underLying pre-Eclampsia) Vital Signs Alert device—designed specifically to improve maternity care in low resource settings—had varying impact when trialled in different countries. To better understand the contextual factors that may contribute to this variation, this study retrospectively evaluated the adoption of CRADLE, during scale-up in Sierra Leone. Methods This was a mixed methods study. A quantitative indicator of adoption (the proportion of facilities trained per district) was calculated from existing training records, then focus groups were held with ‘CRADLE Champions’ in each district (n = 32), to explore adoption qualitatively. Template Analysis was used to deductively interpret qualitative data, guided by the NASSS (non-adoption, abandonment, scale-up, spread, sustainability) Framework. Findings Substantial but non-significant variation was found in the proportion of facilities trained in each district (range 59–90%) [X2 (7, N = 8) = 10.419, p = 0.166]. Qualitative data identified complexity in two NASSS domains that may have contributed to this variation: ‘the technology’ (for example, charging issues, difficulty interpreting device output and concerns about ongoing procurement) and ‘the organisation’ (for example, logistical barriers to implementing training, infighting and high staff turnover). Key strategies mentioned to mitigate against these issues included: transparent communication at all levels; encouraging localised adaptations during implementation (including the involvement of community leaders); and selecting Champions with strong soft skills (particularly conflict resolution and problem solving). Conclusions Complexity related to the technology and the organisational context were found to influence the adoption of CRADLE in Sierra Leone, with substantial inter-district variation. These findings emphasise the importance of gaining an in-depth understanding of the specific system and context in which a new healthcare technology is being implemented. This study has implications for the ongoing scale-up of CRADLE, and for those implementing or evaluating other health technologies in similar contexts. Supplementary Information The online version contains supplementary material available at 10.1186/s12978-022-01551-2.


Maternal mortality in Sierra Leone
In 2020, the maternal mortality ratio (MMR) in Sierra Leone (SL) was estimated to be one of the highest in the world (1120 maternal deaths per 100,000 live births)almost six times the global average [1]. Severe bleeding, infection and blood pressure disorders are the most common causes of maternal deaths, both in Sierra Leone, and globally [2,3]. These complications are correlated with abnormal vital signs, which if detected early, can be managed with simple interventions. Accurately measuring vital signs is therefore critical [4]. However, in SL there is a dearth of critical monitoring equipment and senior healthcare staff, inhibiting timely identification and management of complications [5,6]. There is therefore great capacity to benefit from improved vital signs monitoring in SL.

The CRADLE vital signs alert
The CRADLE Vital Signs Alert (VSA) is a hand-held, upper-arm, semiautomated device, developed to accurately measure women's vital signs throughout pregnancy and the postpartum period. The device measures blood pressure and heart rate, calculates shock index (pulse divided by systolic blood pressure), and digitally displays the results alongside a traffic light early warning system. A green light is displayed if the woman is at low risk of shock or high blood pressure, amber if she needs careful monitoring, or red if she requires emergency treatment. This early warning system is based on comparison of the calculated shock index against set thresholds that have been extensively validated for use in low-resource settings [7,8]. An arrow pointing up or down is also displayed alongside a red or amber light, to indicate whether the issue is related to high or low blood pressure, further guiding the user on appropriate clinical management. The device was designed to be usable by any cadre of health-care provider, including those with little formal training (such as community healthcare workers) and has been specifically tailored for use in low-resource settings; it is low-cost, robust, portable, requires infrequent calibration and has low power requirements [9].
A previous qualitative evaluation of the CRADLE VSA found the device to be well accepted by both healthcare workers and service users, across a range of countries and healthcare settings. Between 2014 and 2016, 155 interviews and six focus groups were undertaken with healthcare workers, pregnant women, and their families, at sites trialling the device across India, Mozambique, Nigeria, and South Africa. Most healthcare workers reported that the device was easy to use and perceived it to be accurate, whilst pregnant women unanimously liked the device, reporting that it increased their understanding of vital signs monitoring [10].
Between 2016 and 2017, CRADLE was introduced across eight low-and middle-income countries, as part of a pragmatic, stepped-wedge, cluster-randomised controlled trial (CRCT) [11] . Post intervention, there was an 8% reduction in primary composite outcome (at least one of eclampsia, emergency hysterectomy and maternal death). However, due to substantial variability within and between clusters, no significant benefit could be attributed directly to the intervention. A concurrent mixedmethods process evaluation found considerable variation in the implementation, reach, adoption, and context between sites, which may have contributed to variation in impact [12], although no significant association was found between the composite 'implementation strength score' and the primary outcome (OR 0.93; 0.07-13.01).

Purpose of this evaluation
Promising technological innovations in healthcare are often hindered by problems of non-adoption or abandonment by individuals and/or difficulties with scale-up and spread within organisations [13]. Provision of the CRADLE device and training package is therefore not a solution in itself, and consideration must be given to how well the device is adopted within the context of a specific healthcare system. Due to the variation in impact and process seen in the CRCT, further in-depth evaluation at individual sites was recommended [12]. Sierra Leone has been the first country to implement CRADLE at scale, providing a unique opportunity to evaluate the adoption of CRADLE outside of a clinical trial setting, where realworld contextual issues may be more apparent.
This retrospective evaluation of the CRADLE scale-up in SL is supported by the evidence-based NASSS (nonadoption, abandonment, scale-up, spread, sustainability) Framework-developed to help predict and evaluate the success of technology-supported health interventions [14]. The Framework consists of seven domains: the technology; the value proposition; the adopter system; the organisation; the wider system; and embedding and adaptation over time. If an intervention is found to have complexity in several domains (i.e. the domains are dynamic, unpredictable and/or are not easily disaggregated into constituent components) an intervention is considered less likely to be adopted and sustained. Guided by the NASSS framework, this evaluation aims to: • Identify any inter-district variations in adoption of CRADLE in SL • Explore the factors impacting upon adoption • Provide guidance on how to avoid non-adoption and abandonment

Context and intervention
Sierra Leone has a fragmented healthcare system, hindered by its colonial history, a long civil war (1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002) and the 2013 Ebola outbreak [15]. The quality of maternal health services is poor, with few women receiving expected standards of care [16]. The CRADLE device and training package was scaled-up in eight of SL's sixteen districts, between March 2020 and January 2021, using a 'training-of-trainers' model [17]. National obstetric experts trained five Master Trainers, who together trained twenty CRADLE Champions (healthcare providers asked to incorporate training into their existing roles) per district. Champions were given materials and asked to disseminate training at all Ministry of Health and Sanitation (MoHS) facilities within a designated catchment area. Implementation districts were selected by the MoHS and included both urban and rural areas.

Methods and analysis
The evaluation took a pragmatic, mixed methods approach-as recommended for the evaluation of health programmes [18].

Quantitative
An existing database of training registers, held by the implementing organisation, was used to calculate the proportion of facilities trained per district. This served as a proxy of adoption (by both the CRADLE Champions and the facilities offered training). This information was used to guide the selection of districts for the qualitative evaluation, aiming to choose a sample most representative of the spread of adoption levels. A chi-squared goodness-of-fit test was also performed to test whether inter-district differences in adoption were statistically significant.

Qualitative
Focus Group Discussions (FGDs) were held with Champions in the four selected districts, to explore their views and experiences of the CRADLE intervention and reflections on how it was adopted by others. Around half of the Champions in each district were invited to participate (total invited n = 41). Champions were purposefully selected to represent the range of Champion demographics. A topic guide was used flexibly and questions openended, to maximise breadth of responses and allow for unanticipated adoption-related topics. Each FGD had two moderators, one Sierra Leonean (BS) and one British (SB), both female. Discussions were in English, but use of local languages was permitted where necessary to convey meaning. Discussions were audio-recorded, and field notes taken. Qualitative data were interpreted via Template Analysis [19]. Analysis was primarily deductive, using the domains of the NASSS Framework as a priori themes. However, development of the coding Framework was an iterative process, remaining open to the refinement or removal of themes where appropriate. A subtle-realist stance underpinned the analysis i.e. the belief that the researcher's analysis is influenced by their social stance, but that phenomena are to an extent knowable through the research process [20].

Quantitative
The proportion of facilities trained varied from 59 to 90% per district (mean 77.5%; Table 1). This variation, although substantial, was not statistically significant [χ 2 (7, N = 8) = 10.419, p = 0.166]. Four districts with varying adoption levels, including those with the highest (D, H) and lowest (A, B) adoption, were invited to participate in FGDs. The districts aren't named to ensure participant confidentiality.

Qualitative
FGDs (n = 4), lasted 109-141 min. The time between initial training and FGDs was 8-12 months, giving all districts adequate implementation time. Participation rate was 78%, with at least seven participants per district. Participant demographics are presented in Table 2. The majority were Female (94%), Midwives (78%), and based at Community Health Centres (53%).
Using Template Analysis, all codes aligned with one of the seven NASSS domains. The two most dominant domains were 'The Technology' (152 references) and 'The Organisation' (127 references). The number of references made per code were disaggregated by district, to demonstrate inter-district variations in the comments made (see Additional file 1).
Based on the qualitative findings, each domain was classified as 'simple' , 'complicated' or 'complex' , in  accordance with the Framework (see Additional file 2). Two domains-'The Technology' and 'The Organisation'-were classified as 'complex' , posing a substantial threat to adoption. The technology was considered to be complex because of reported charging issues (including battery faults), concerns about ongoing procurement, and potential to doubt or misinterpret results if inadequately trained. The organisation was considered to be complex because of infighting, high staff turnover, and the extensive work involved in implementation. These domains are also interdependent, for example, the need for ongoing training increases the work involved in implementation.
Providing the device, and training staff in its use, is therefore not in itself enough to ensure success. Several factors stood out as being more prevalent in districts with lower adoption: power struggles (mainly in District A), a lack of trust in the MoHS (only in District B) and mistrust in device accuracy (both A and B). Each reference included under the NASSS domains was subsequently coded as either a barrier or facilitator to adoption. The proportion of references coded as barriers was greater in districts with low adoption than those with high adoption (46%, 40%, 31% and 33% for Districts A, B, H and D, respectively). This may reflect more barriers faced, or a different mentality towards challenges.

The condition
Several participants commented that pregnant women delay coming to government health facilities, often first seeking advice or traditional medicine elsewhere-also reported by pregnant women in other studies [21]. This may contribute to women presenting with complex, urgent needs.
The woman had delivered 5 days and she was brought to clinic, they said she saw devil […] When they came with that woman at the facility, when they came with her at the facility she was convulsing, and they had put garlic and others over her.-District A However, participants reported that CRADLE helps mitigate against late presentation by supporting prompt action.
"The problem we were having, the pregnant women they delay to come to the health facilities because they need to seek advice from their, from their inlaws or from their elderly people in the villages. […] But with the help of the CRADLE machine, when she comes, immediately you will be able to monitor and detect whatever problem.-District B

The technology
Participants across all districts reported that CRADLE was easy to use and improved the accuracy of vital signs recording.
Because at first, they used to just give the, the vital signs that they feel like giving, but because of the CRADLE they now give us correct vital signs.-District H Incorrect use (e.g. poor positioning) could raise doubts about its accuracy. However, this is eased with ongoing support. Doubts about accuracy were more common in districts with low adoption (A and B).
[ Concerns were also raised about the ongoing procurement of spare devices and parts (unavailable in SL). Participants worried about becoming reliant on a device that may subsequently be withdrawn. These concerns were mentioned more frequently in low-adoption districts.
Like this, good things are brought in this country, so my worry is, in case we are used to this one and then this thing phase out, then there is nowhere we can get this machine, how are we going to deal again with our pregnant women? So this was my worry, that came in to my mind.

The value proposition
Champions in all districts unanimously reported that the device was desirable and effective. In keeping with the previous multi-site evaluation [12], the device was reported to support problem identification and management and met an urgent need for blood pressure machines. A novel finding is that Champions reported noticing a reduction in maternal deaths since using the device, with several providing personal accounts of this.

The organisation
Participants identified several barriers to implementation at facility level. Firstly, it was hard for Champions to access facilities, particularly during raining season, and the associated cost often exceeded the amount of money they had for travel. Many reported making sacrifices to overcome these barriers because they felt compelled to supply the device to others.

I had to take bike for the rest of the day [unintelligible], but it's because of I like the job and that makes me Champion, because I have to sacrifice. I have
to give more so that the next facility will gain then they will have their CRADLEs. If you look, look for the amount, it's too small. But we sacrifice so that the job will go well. So that's why we spent a lot of money.-District D High staff turnover within the MoHS was also identified as a challenge, as Champions were often the ones training new staff. Follow up and encouragement from the District Health Management Team (DHMT) and implementing organisation was reported to motivate Champions to persist in spite of these challenges.

She was motivating us to continue with the training and it is very good because to be frank when some-body is there to say, 'continue doing this, continue doing this' , you will hold on to it the more and you will continue to do it.-District H
At facility level, power struggles between Champions and trainees were commonly reported, particularly where normal hierarchy was challenged. Internal professional conflicts could also deter staff from attending training, even when the conflict did not directly involve the Champion. These issues were mentioned by all districts but primarily by District A (lowest adoption).

Embedding and adaptation over time
A few Champions doubted that the MoHS could sustain the use of CRADLE independently and several commented that continuation of external support was needed (predominantly District B-low adoption).
Because we know the government of, our government, sometimes something will come out very, very good. It will start good, like very speed, it will just die out slowly, slowly. We pray that this will not happen to CRADLE machine.-District B We have similar things like for the drugs […] You will requisite, at the end of the day they will tell you the drug is not available […] continue to report, continue to report and there is no action, this is our fear.-District B However, more Champions, across all districts, felt they could take ownership of sustainability efforts-integrating with existing activities and advocating to the MoHS.

Adoption of CRADLE in SL
Based on quantitative adoption indicators, average adoption levels in this study were lower than seen in Freetown during the CRCT (77.5% versus 96.2%). This likely reflects differences in the adoption indicator used: 'percentage of facilities trained' in this study versus 'percentage of clinical areas using solely the CRADLE at 12 months' in the CRCT. The former reflects whether Champions offered training and the facility accepted it (i.e. engagement with the intervention), whilst the latter measures use of the innovation post-training. The difference in adoption levels may therefore reflect that whilst staff are willing and able to adopt the device once adequately trained and supported, barriers to implementation during scale-up may hinder engagement at the outset. Furthermore, adoption was not uniform across all districts, with some districts reaching adoption levels of 90%. Exploring this variation has helped identify factors that may contribute to nonadoption and potential strategies to overcome them.

Complexity impacting upon adoption
In the previous evaluation of the CRADLE CRCT, few sites mentioned barriers to adoption, primarily citing the device's sensitivity to movement [12]. In contrast, this study identified many barriers, with substantial complexity identified around the technology and the organisation. Potential strategies to manage these are discussed, but it is important to note that complexity cannot be completely solved and may even increase with time [22]. Periodic monitoring throughout the scale-up in SL is therefore recommended to help manage emerging complexities. This may be supported by practical, purposedesigned tools, such as the NASSS-CAT Tools [23].

Organisational factors
Several barriers to implementation at facility level were identified by Champions. The district with lowest levels of adoption mentioned power struggles and infighting frequently, and far more than other districts. Intra-and inter-professional conflicts in healthcare settings have been reported elsewhere in Sub-Saharan Africa in relation to staff burnout [24], but not previously in relation to implementation of health programmes. This study suggests that professional relationships may influence health technology adoption in Sierra Leone, and so should be considered when designing, implementing, and evaluating new programmes in this context. To help avoid professional conflicts, Champions reported that initial consultation with facilities (e.g. about the programme, the plan for training, and the lack of financial incentives) must be communicated transparently and carefully. To respect existing hierarchies, this information may be best relayed by someone in a position of authority (e.g. a member of the DHMT), and/or Champion selection should consider candidate's inter-personal skills.
Some Champions independently took the initiative to engage community leaders in their area. These Champions reported that the community leaders assisted them with implementation and enabled them to gain the support of other healthcare providers and the wider community. Involving Community Leaders in Public Health interventions is widely encouraged [25] and was critical in controlling the 2013 Ebola outbreak in SL [26]. This strategy could therefore be actively promoted to CRA-DLE Champions in ongoing scale-up efforts.

Technological factors
In keeping with earlier CRADLE studies [10,12,27], respondents generally found the device easy to use, but small errors in application (e.g. incorrect positioning) could lead to staff mistrust. A novel finding was that interpretation of the arrows (critical for guiding clinical management), was difficult for some staff. Modifying this aspect of the training package-co-creating new materials with staff who found it challenging-may help to ensure effectiveness [28]. Champions also reported using their own strategies to support training of staff (e.g. WhatsApp). Rather than ensuring implementation fidelity, these sorts of initiatives and contextually appropriate modifications should continue to be encouraged throughout scale-up to promote adoption [29].
Difficulty charging the device was mentioned frequently and could lead to abandonment if not addressed. Approximately 40% of maternal and child health posts in Sierra Leone report having no electricity source [30], and previous studies have commented that unstable power supplies can limit technological adoption in Sub-Saharan Africa [31]. CRADLE Champions found novel ways to manage this issue in the short term, such as charging devices outside of the health facility. Further, 'Solar suitcases'-easy-to-use solar electric systems that can power small medical devices-are being rolled out in remote maternal health facilities nationally and may help mitigate against this issue to an extent [32]. However, in the long term, providing a stable power supply to all health facilities is a clear priority. Champions also reported faults with the battery itself (e.g., not retaining charge), which were similarly reported in the CRCT [11]. The mechanisms of these faults are yet unknown and Medical Equipment Technicians (Ministry of Health and Sanitation staff who have been familiarised with the device) are now undertaking further investigation into the issue. In the interim, it will be important to ensure that training on device maintenance, including correct charging practices, remain part of the CRADLE training package.
Finally, concerns were also raised about the MoHS' capacity to sustain the programme, particularly in relation to procurement of devices and parts. This may reflect fragmentation in the healthcare system and disparity between those holding decision-making power and resources (central government, external donors) and those responsible for delivering services at district level [15]. Several steps were taken during implementation to promote sustainable training and procurement, including adding the device to the National Health Facility Equipment List, and incorporating CRADLE within the University obstetric curricula and the National Emergency Obstetric and Newborn Care (EmONC) training programme. Informing Champions and District Management about such actions, and keeping them informed regarding ongoing procurement, may help to allay sustainability concerns.

Transferability of the NASSS framework
Empirical work to develop NASSS and subsequent applications has been conducted entirely in high-income settings [14,33]; this study was the first to test its utility in a low-income setting. All qualitative data codes in this study aligned with the NASSS domains, suggesting the framework translates to low-income contexts. However, further application in other low-income contexts should be conducted to support or refute this. As professional conflict was identified as a threat to adoption in this context, it may be valuable to add questions about this to the NASSS guidance.

Limitations
FGDs were held in only four districts; completing focus groups in other districts may have yielded different results. However, when coding data, no new codes emerged after analysis of the third FGD, suggesting data saturation was reached. The use of FGDs may have increased the risk of social desirability bias. However, Champions reported many barriers, and conflicting opinions were offered within the same FGDs, suggesting participants felt able to speak openly. The evaluation is specific to the adoption of CRADLE intervention and SL, limiting the generalisability of findings. However, contextual information has been provided to allow readers to draw from the findings where appropriate. The author was the Programmes Manager of the implementing organisation of CRADLE during this phase of scale-up. There are potential benefits of this 'embedded implementation research' approach, including better contextual understanding of the findings [34]. However, this also increased the risk of investigator bias. To mitigate this, questions were designed to be non-leading, and the analysis reflective of the data. The author is also White British, which in light of the colonial history of SL may have created a power imbalance limiting participants' willingness or ability to voice their opinions [35]. To mitigate this, the lead moderator of FGDs was a Sierra-Leonean Midwife, known to the Champions.

Conclusion
This study found considerable inter-district variation in adoption of the CRADLE device in SL. Complexity was identified in two NASSS domains, 'the technology' and 'the organisation' with barriers to adoption including infighting, high staff turnover, charging issues, and concerns about ongoing procurement. Device provision and training are therefore unlikely to be enough to ensure its sustained adoption and long-term impact. Strategies identified to mitigate against issues and promote sustained adoption in SL include: early, transparent communication to facilities about training plan (e.g. lack of financial incentives, rationale for breaking hierarchies where applicable); selecting Champions with strong softskills (particularly conflict resolution, problem solving and tenacity); encouraging localised adaptations during implementation (including involvement of relevant community leaders); further investigation into and resolution of device charging issues; and informing all cadres of staff of updates regarding ongoing device and parts procurement. This study has implications for the ongoing scaleup of the CRADLE in SL and for those implementing or evaluating similar technology innovations in the region.